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Abstract 



For the Hermitian inexact Rayleigh quotient iteration (RQI), we present a new 
general theory, independent of iterative solvers for shifted inner linear systems. The 
theory shows that the method converges at least quadratically under a new condition, 
called the uniform positiveness condition, that may allow inner tolerance ^fc > 1 at 
outer iteration k and can be considerably weaker than the condition ^fc < ^ < 1 with ^ 
a constant not near one commonly used in literature. We consider the convergence of 
the inexact RQI with the unpreconditioned and tuned preconditioned MINRES method 
for the linear systems. Some attractive properties are derived for the residuals obtained 
by MINRES. Based on them and the new general theory, we make a more refined 
analysis and establish a number of new convergence results. Let |jrfc|| be the residual 
norm of approximating eigenpair at outer iteration k. Then all the available cubic and 
quadratic convergence results require £_k — 0(||rfe||) and ^fe < ^ with a fixed ^ not near 
one, respectively. Fundamentally different from these, we prove that the inexact RQI 
with MINRES generally converges cubically, quadratically and linearly provided that 
£.k < £. with a constant ^ < 1 not near one, ^fc = 1 — 0(||rfe||) and ^fc = 1 — 0(||rfe|p), 
respectively. Therefore, the new convergence conditions are much more relaxed than 
ever before. The theory can be used to design practical stopping criteria to implement 
the method more effectively. Numerical experiments confirm our results. 
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1 Introduction 

We consider the problem of computing an eigenvalue A and the associated eigenvector x of 
a large and possibly sparse Hermitian matrix A G C""^", i.e., 



Throughout the paper, we are interested in the eigenvalue Ai closest to a target a and its 
corresponding eigenvector xi in the sense that 
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Ax = Ax. 




Ai — (t| < IA2 — 0"! < • • • < |A„ — a\. 
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Suppose that a is between Ai and A2. Then we have 

|Ai < ^|Ai -A2I. (3) 

We denote by xi, X2, • • • , a;„ are the unit length eigenvectors associated with Ai, A2, • • • , A,i. 
For brevity we denote (Ai,xi) by (A,x). There are a number of methods for computing 
(A, x), such as the inverse iteration [15], the Rayleigh quotient iteration (RQI) [15j, the 
Lanczos method and its shift-invert variant [15], the Davidson method and the Jacobi- 
Davidson method [19[[21]. However, except the standard Lanczos method, these methods 
and shift-invert Lanczos require the exact solution of a possibly ill-conditioned linear system 
at each iteration. This is generally very difficult and even impractical by a direct solver since 
a factorization of a shifted A may be too expensive. So one generally resorts to iterative 
methods to solve the linear systems involved, called inner iterations. We call updates of 
approximate eigenpairs outer iterations. A combination of inner and outer iterations yields 
an inner-outer iterative eigensolver, also called an inexact eigensolver. 

Among the inexact eigensolvers available, the inexact inverse iteration and the inexact 
RQI are the simplest and most basic ones. They not only have their own rights but also are 
key ingredients of other more sophisticated and practical inexact solvers, such as inverse 
subspace iteration [14j and the Jacobi-Davidson method. So one must first analyze their 
convergence. This is generally the first step towards better understanding and analyzing 
other more practical inexact solvers. 

For A Hermitian or non-Hermitian, the inexact inverse iteration and the inexact RQI 
have been considered, and numerous convergence results have been established in many 
papers, e.g., [B [H H [71 [H HH [HI [HI ig [20l |22] and the references therein. For the 
Hermitian eigenproblem, general theory for the inexact RQI can be found in Berns-Miiller 
and Spence [T], Smit and Paadekooper [T8] and van den Eshof [20]. They prove that the 
inexact RQI achieves cubic and quadratic convergence with decreasing inner tolerance = 
0{\\rk\\) and ^a: < C < 1 with a constant ^ not near one. Supposing that the shifted linear 
systems are solved by the minimal residual method (MINRES) [13[ I16j . mathematically 
equivalent to the conjugate residual method [T6], Simoncini and Elden |17j prove the cubic 
and quadratic convergence of the inexact RQI with MINRES and present a number of 
important results under the same assumption on ^fc. Simoncini and Elden first observed 
that the convergence of the inexact RQI may allow to almost stagnate, that is, i.e., 
near one. Xue and Elman [22] have refined and extended some results due to Simoncini and 
Elden. They have proved that MINRES typically exhibits a very slow residual decreasing 
property (i.e., stagnation in their terminology) during initial steps but the inexact RQI 
may still converge; it is the smallest harmonic Ritz value that determines the convergence 
of MINRES for the shifted linear systems. Furthermore, although Xue and Elman's results 
have indicated that very slow MINRES residual decreasing may not prevent the convergence 
of inexact RQI, too slow residual decreasing does matter and will make it fail to converge. 
Besides, for the inexact RQI, to the author's best knowledge, there is no result available on 
linear convergence and its conditions. 

In this paper we first study the convergence of the inexact RQI, independent of iterative 
solvers for inner linear systems. We present new general convergence results under a certain 
uniform positiveness condition, which takes into account the residual directions obtained 
by iterative solvers for the inner linear systems. Unlike the common condition ^fc < ^ < 1 
with ^ a constant, it appears that the uniform positiveness condition critically depends on 
iterative solvers for inner iterations and may allow ^fc ~ 1 and even > 1, much weaker 
than the common condition in existing literatures. 

We then focus on the inexact RQI with the unpreconditioned MINRES used for solving 
inner shifted linear systems. Our key observation is that one usually treats the residuals 
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obtained by MINRES as general ones, simply takes their norms but ignores their directions. 
As will be clear from our general convergence results, residual directions of the linear systems 
play a crucial role in refining convergence analysis of the inexact RQI with MINRES. We first 
establish a few attractive properties of the residuals obtained by MINRES for the shifted 
linear systems. By combining them with the new general convergence theory, we derive a 
number of new insightful results that are not only stronger than but also fundamentally 
different from the known ones in the literature. We show how the inexact RQI with MINRES 
meets the uniform positiveness condition and how it behaves if the condition fails to hold. 

As will be clear, we trivially have ^^fe < 1 for MINRES at any inner iteration step. 
We prove that the inexact RQI with MINRES generally converges cubically if the uniform 
positiveness condition holds. This condition is shown to be equivalent to < ^ with a 
fixed ^ not near one, but the inexact RQI with MINRES now has cubic convergence other 
than the familiar quadratic convergence. Cubic convergence does not require decreasing 
inner tolerance ^fc = Odl^fcH) any more. We will see that ^ = 0.1, 0.5 work are enough, 
^ = 0.8 works well and a smaller ^ is not necessary. We prove that quadratic convergence 
only requires = 1 — 0(||rfc||), which tends to one as ||rfc|| — )• and is much weaker than 
the familiar condition ^fc < C with a constant ^ not near one. Besides, we show that a linear 
convergence condition is = 1 — 0(||rfc|p), closer to one than 1 — 0(||rfc||). Therefore, 
if stagnation occurs during inner iterations, the inexact RQI may converge quadratically 
or linearly; if stagnation is too serious, that is, if is closer to one than 1 — 0(||rfc|p), 
the method may fail to converge. Note that, for the inner linear systems, the smaller 
is, the more costly it is to solve them using MINRES. As a result, in order to achieve 
cubic and quadratic convergence, our new conditions are more relaxed and easier to meet 
than the corresponding known ones. Therefore, our results not only give new insights 
into the method but also have impacts on its effective implementations. They allow us to 
design practical criteria to best control inner tolerance to achieve a desired convergence rate 
and to implement the method more effectively than ever before. Numerical experiments 
demonstrate that, in order to achieve cubic convergence, our new implementation is about 
twice as fast as the original one with = 0(||rfc||). 

Besides, we establish a lower bound on the norms of approximate solutions Wk+is of the 
linear systems obtained by MINRES. We show that they are of 0{ )) ^^'^ 
when the inexact MINRES converges cubically, quadratically and linearly, respectively. So 
lltffc+ill can reflect how fast the inexact RQI converges and can be used to control inner 
iteration, similar to those done in, e.g., [T71[22]. Making use of the bound, as a by-product, 
we present a simpler but weaker convergence result on the inexact RQI with MINRES. It 
and the bound for Hw^+iH are simpler and interpreted more clearly and easily than those 
obtained by Simoncini and Elden jl7j . However, we will see that our by-product and their 
result are weaker than our main results described above. An obvious drawback is that 
the cubic convergence of the exact RQI and of the inexact RQI with MINRES cannot be 
recovered when ^fc = and = respectively. 

It appears [ll|5l[22] that it is often beneflcial to precondition each shifted inner linear sys- 
tem with a tuned preconditioner, which can be much more effective than the corresponding 
usual preconditioner. How to extend the main results on the inexact RQI with the unpre- 
conditioned MINRES case to the inexact RQI with a tuned preconditioned MINRES turns 
out to be nontrivial. We will carry out this task in the paper. 

The paper is organized as follows. In Section [21 we review the inexact RQI and present 
new general convergence results, independent of iterative solvers for the linear systems. In 
Section [3l we present cubic, quadratic and linear convergence results on the inexact RQI 
with the unpreconditioned MINRES. In Section HJ we show that the theory can be extended 
to the tuned preconditioned MINRES case. We perform numerical experiments to confirm 
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our results in Section [5l Finally, we end up with some concluding remarks in Section [6l 

Throughout the paper, denote by the superscript * the conjugate transpose of a matrix 
or vector, by / the identity matrix of order n, by || • || the vector 2-norm and the matrix 
spectral norm, and by Amim Amax the smallest and largest eigenvalues of A, respectively. 



2 The inexact RQI and general convergence theory 

RQI is a famous iterative algorithm and its locally cubic convergence for Hermitian problems 
is very attractive [15]. It plays a crucial role in some practical effective algorithms, e.g., 
the QR algorithm, [6l [15]. Assume that the unit length u/^ is already a reasonably good 
approximation to x. Then the Rayleigh quotient 9k = ul.Auk is a good approximation to 
A too. RQI computes a new approximation u^+i to x by solving the shifted inner linear 
system 

{A - ekl)w = Uk (4) 

for Wk+i and updating u^^i = if^/t+i/||ii'fc+i || and iterates until convergence. It is known 
P da [E] that if 

|A — 6'ol<- min |A — Aol 

2 i=2,3,...,n ' 

then RQI (asymptotically) converges to A and x cubically. So we can assume that the 
eigenvalues of A are ordered as 

\x-ek\<\X2-9k\<---<\Xn-ek\. (5) 

With this ordering and Amin < < Amax, we have 

\X - 9k\ < \\\ - X2\. (6) 

An obvious drawback of RQI is that at each iteration k we need the exact solution Wk+i 
of {A — 9kl)w = Uk- For a large A, it is generally very expensive and even impractical to 
solve it by a direct solver due to excessive memory and/or computational cost. So we must 
resort to iterative solvers to get an approximate solution of it. This leads to the inexact 
RQI. ([4]) is solved by an iterative solver and an approximate solution Wk+i satisfies 

{A - 9kl)wk+i = Uk + (,kdk, Uk+i = Wk+i/\\wk+i\\ (7) 

with < < ^, where £,kdk with \\dk\\ = 1 is the residual of (A — 9kl)w = Uk, is the 
residual direction vector and is the relative residual norm (inner tolerance) as = 1 
and may change at every outer iteration k. This process is summarized as Algorithm 1. If 
= for all /c. Algorithm 1 becomes the exact RQI. 

It is always assumed in the literature that ^ < 1 when making a convergence analysis. 
This requirement seems very natural as heuristically (jH) should be solved with some accu- 
racy. Van den Eshof |20j presents a quadratic convergence bound that requires ^ not near 
one, improving a result of |18j by a factor two. Similar quadratic convergence results on the 
inexact RQI have also been proved in some other papers, e.g., [U |T71 [H], under the same 
condition on ^. To see a fundamental difference between the existing results and ours (cf. 
Theorem [2|), we take the result of [20] as an example and restate it. Before proceeding, we 
define 

n '^max Amin 

' IA2-AI 

throughout the paper. We comment that Amax — Amin is the spectrum spread of A and 
IA2 — A| is the gap or separation of A and the other eigenvalues of A. 
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Algorithm 1 The inexact RQI 



1: Choose a unit length uq, an approximation to x. 

2: for fc=0,l, ... do 

3: 9k = ulAuk. 

4: Solve {A — 9kl)w = Uk for Wk+i by an iterative solver with 

II - 0kI)Wk+l -Uk\\= Ck< 



5: Uk+1 = Wk+l/\\wk+l\\. 

6: If convergence occurs, stop. 
7: end for 



Theorem 1. [20j Define (fik = Z(ufc,x) to he the acute angle between andx, and assume 
that Wk+i is such that 

\\{A- 9kl)wk+i -Uk\\=(k<^<l. (8) 

Then letting (pk+i = '^{uk+i^x) be the acute between u^+i andx, the inexact RQI converges 
quadratically: 

tan(/>fc+i < , sin^ 0^ + 0(sin^ (pk)- (9) 

vi - " 

However, we will see soon that the condition ^ < 1 not near one can be stringent and 
unnecessary for quadratic convergence. To see this, let us decompose Uk and dk into the 
orthogonal direct sums 

■Ufc = X cos(?;>fc + efc sin</)fc, _L x, (10) 
dk = X cos ^k + fk simpk, fk-^x (11) 

with ||efc|| = ll/fcll = 1 and ipk = ^(dk^^)- Then ([7]) can be written as 

[A - 9kl)wk+i = {cos (pk + Ck cosV'fc)x + (efc siiKpk + ^kfk simpk). (12) 

Inverting A — 9kl gives 

Wk+i = {X - 9 k)'^ {cos (pk + Ck cosi;k)x + {A-9kiy^{ek sm4>k + Ckfk sin^-fc). (13) 

We now revisit the convergence of the inexact RQI and prove that it is the size of 
I cos <j)k + Ck cosV'fcl other than ^fc < ^ < 1 that is critical in affecting convergence. 

Theorem 2. // the uniform positiveness condition 

\ cos (j)k + Ck cos ipk\ > c (14) 

is satisfied with a constant c > uniformly independent of k, then 

, , / o« sin (t>k + £,k sin il^k . 2 , 

tan0fe+i < 2(j- — — — fsm cpk (15) 

I cos (pk + ik cosipk] 

< ^sin2 0^ + O(sin3 0fc), (16) 

that is, the inexact RQI converges quadratically at least for uniformly bounded (,k 1^ ^ with 
^ some moderate constant. 
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Proof. Note that (113p is an orthogonal direct sum decomposition of Wk+i since for a Her- 
mitian A the second term is orthogonal to x. We then have 

, , 1^ .\\{A- Okiy^iek sincpk + Ckfk simpk)\\ 
tan0fc+i = |A — Uk\ 



\ cos (j)k + ^k cosV'fcl 
As A is Hermitian and -L x, it is easy to verify (cf. |15l p. 77]) that 

X-Ok = {X- elAck) SVC? (pj,, 

IA2 — A| < |A — elAekl < Amax — Amin, 

IA2 - A| sin^ (pk <\X- 9k\ < (Amax - Amin) sin^ (j)k- (17) 

Since 

\\{A-ekiy\ek sm<j)k+^kfk sinV/c)|| < - 0fc/)"^efe|| sin(/)fc + ^felKA - 0fe/)"Vfc|l sinV'fe 

< IA2 - 6'fc|"^(sin0fe +^fesiii'0fc) 

< 2\X2 - X\^^ {sin (j)k + ^ksmipk) 

with the last inequality holding because of ([5]), we get 

2 (sin (f)k + Cfc siu'^fc) 



tan(/)/c_^_i < |A — e^^Cfclsin 



IA2 - A|| cos + S,k cosipkl 



2|A„-A| sin 0fc + ^fc sin . 2 , 
IA2 - A| I cos^fc + CfcCosV'fcl 

^ ^ 2(Amax ~ '^min) -21 , ,0/ • 3 j, \ 

- u 4>k + (sm^ (f)k ) . 

c|A2 - A| 



□ 



Define ||rfc|| = ||(^ — 9kl)uk\\- Then by ([6]) we get IA2 — Ok\ > ^2^-^. So it is known 



from [15'. Theorem 11.7.1] that 



< sin^fc < if^^^^'^L (18) 



Amax ~ Amin 1^2 — A| 

We can present an alternative of ()16p when ||rfc|| is concerned. 
Theorem 3. // the uniform positiveness condition (jl4p holds, then 



'\k 



\rk+i\\< ,7 + 0(||rfcf). (19) 

c|A2 - A| 



Proof. Note from ([THD that 

Ikfc+il 



< sin0fc+i < tanc/ifc+i. 



Amax Xy 

Substituting it and the upper bound of ([T8|) into ([T6|) establishes (fT9]) . □ 

For the special case that the algebraically smallest eigenvalue is of interest, Jia and 
Wang [To] proved a slightly different result from Theorem [2j Similar to the literature, 
however, they still assumed that ^fc < ^ < 1 and did not analyze the theorem further, 
though their proof did not use this assumption. A striking insight from the theorem is that 
the condition ^a: < < 1 may be considerably relaxed. If cosipk is positive, the uniform 
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positiveness condition holds for any uniformly bounded ^fc < ^. So we can have ^ > 1 
considerably. If cos ipk is negative, then | cos (pk + S,k cos ipk\ ^ c means that 

C-COS</>fc 

COS ipk 

if COS (pk + S,k COS ipk ^ c with c < 1 is required or 

C + cos 

4fc > ; — 

- cos Ipk 

if —cos(j)k — ^fccos^fc > c is required. Keep in mind that coscj)^ ~ 1. So the size of 
critically depends on that of cosV'fci ^-nd for a given c we may have 1 and even > 1. 
Obviously, without the information on cos'i/'fc, it would be impossible to access or estimate 
^k- As a general convergence result, however, its significance and importance consist in 
that it reveals a new remarkable fact: It appears first time that (13) may allow to be 
relaxed (much) more than that used in all known literatures and meanwhile preserves the 
same convergence rate of outer iteration. As a result, the condition (,k ^ ^ < ^ with 
constant ^ not near one may be stringent and unnecessary for the quadratic convergence of 
the inexact RQI, independent of iterative solvers for the linear systems. The new condition 
has a strong impact on practical implementations as we must use a certain iterative solver, 
e.g., the very popular MINRES method and the Lanczos method (SYMMLQ) for solving 
{A — OkI)w = Uk- We will see that costpk is critically iterative solver dependent. For 
MINRES, cosV'A: liS'S some very attractive properties, by which we can precisely determine 
bounds for in Section [3l which are much more relaxed than those in the literature. For 
the Lanczos method, we refer to [9] for cos ipk and ^fc, where cos ipk and S,k £ire fundamentally 
different from those obtained by MINRES and ^fc > 1 considerably is allowed. 

Remark 1. If = for all k, the inexact RQI reduces to the exact RQI and The- 
orems [2H3] show (asymptotically) cubic convergence: tan(j)k-i^i = 0{sm^ (/)k) and = 
OiWrkf). 

Remark 2. If the linear systems are solved with decreasing tolerance ^k = 0(sin(^fc) = 
0(||rfc||), then tan(/)fc_|_i = 0{sm^ (/)k) and ||rfc+i|| = 0(||rfc||^). Such (asymptotically) cubic 
convergence also appears in several papers, e.g., [I1[IH1[2D], either explicitly or implicitly. 



3 Convergence of the inexact RQI with MINRES 

The previous results and discussions are for general purpose, independent of iterative solvers 
for (A — 6kl)w = Uk- Since we have Amin < ^fe < Amax, the matrix A — Okl is Hermitian 
indefinite. One of the most popular iterative solvers for [A — 9kl)w = Uk is the MINRES 
method as it has a very attractive residual monotonic decreasing property \T3[ I16j . This 
leads to the inexact RQI with MINRES. 

We briefly review MINRES for solving At outer iteration k, taking the starting 
vector vi to be Uk, the m-step Lanczos process on j4 — 9kl can be written as 

{A — 6kI)Vm = VmTra + tm+lmVm+lKn = ym+lTra, (20) 

where the columns of Vm = (^i; • • • , Vm) form an orthonormal basis of the Krylov subspace 
lCm{A - 6kI,Uk) = lCm{A,Uk), Vm+i = {Vm,Vm+i), = [Uj] = V^{A - 9kI)Vm and 
Tm = V^^i{A — 9kI)Vm [l5l[16]. Taking the zero vector as an initial guess to the solution 
of {A — 9kl)w = Uki MINRES (BJ [I3l [16] extracts the approximate solution Wk+i = Vmy 
to {A — 9kl)w = Uk from lCm{A.,Uk)-, where y is the solution of the least squares problem 
min ||ei — T^vW with ei being the first coordinate vector of dimension m + 1. 
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By the residual monotonic decreasing property of MINRES, we trivially have < 
\\uk\\ = 1 for all k and any inner iteration steps m. Here we must take m > 1; for 
m = 1, it is easily verified that y = and thus Wk+i = and ^fc = 1 by noting that 
til = ~ dkl)uk = 0. So Ufc+i is undefined and the inexact RQI with MINRES breaks 
down if m = 1. 

The residual direction vectors obtained by MINRES have some attractive features 
and we can precisely get subtle bounds for sin-f/^fc and cosipk: as the following results show. 

Theorem 4. For MINRES, let the unit length vectors and he as in (jlOp and (jlip . 

define the angle (pk = -^{fk, — Qkl)^k) <ind assume that 

I cos yjfc I > I cos v?! > (21) 
holds uniformly for an angle (p away from ^ independent of k, i.e., 

\fl{A-ei)ek\ = \\{A-ekI)ek\\\cos^k\ > \\{A - ekI)ek\\\cosip\. (22) 
Then we have 

simpk < I I sincpk, (23) 
I cos Lfl 

cosV'fc = ±(1-O(sin2 0fc)). (24) 

Furthermore, provided > siiKpk, then 

cosipk = -1 + 0(sin^ (/jfc), (25) 
dk = -X + O {sin (f)k)- (26) 

Proof. Note that for MINRES its residual £,kdk satisfies £,kdk -L (A — 6kI)ICm{A,Uk). There- 
fore, we specially have ^kdk -L {A — OkI)uk, i.e., d^ -L {A — 6kl)uk. Then from (jlOp and (jlip 
we obtain 

iX-6k) cos (pk cos ipk + fl{A- 6kl)ek sin 0^ sin tpk = 0. 

So 

, , {^k - A) cos (t)k 

fl{A-ekI)ekS\ii(t)k 

By |A - ek\ < we get 



\f^{A-9kI)ek\ = ||(^-^fc/)efc|||cos(^fc| 

> ||(^ - ^fc7)efc||| cosv^l 

> IA2 — ^fcll COS(^| 



^ -Al l I 

> |COS(/?|. 



Using pT|) . we obtain from (p7|) 

I tan ipk I ^ 



(Amax - Amin) siu (f)k COS 

|/*(^-0fc/)efc| 

2 (Amax Ajnin) . 

^ "R m r cos (pk 

IA2 - A|| COS(/9| 

< 1 fsm^fc. 

I cos 
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Therefore, ([23l) holds. Note that ([23l) means s'mipk = O(sin0fc). So we get 



cosV-fc = ±Y 1 - sin^ V'fc = ±(1 - -sin^ Vfc) + 0(sm'' Vfc) = ±(1 - O {sin^ 4>k)) 

by dropping the higher order term 0(sin^ (/)k)- 

Now we prove that cosV'fc and cos^fc must have opposite signs if S^k > sin^^. Since the 
MINRES residual 

^kdk = {A- 6kI)'Wk+i - Uk, 

by its residual minimization property we know that (^4 — 6kl)wk+i is just the orthogo- 
nal projection of Uk onto {A — 9kI)K,m{A,Uk) and S,kdk is orthogonal to (^4 — 9kl)wk+i- 
Therefore, we get 

il + \\{A - ekl)wk+if = \\ukf = 1. (28) 

Note that 

[A - ekl)wk+i = Uk + ikdk = (cos (t)k + ik COS ^k)x + (cfc sin (pk + ikfk sin ^k) 
is an orthogonal direct sum decomposition of {A — 9kl)wk+i- Therefore, we have 

\\{A - 9kl)wk+i\? = {cos(pk + Ck cos ipkf + ||efe sin^fc + CklkSimpkH"^, 
which, together with ([28|) . gives 

(cos (j)k + Cfc COS i^kf + Cfc < 1- 

Solving it for ^k cos ipk, we have 

- cos (pk - \J'^-il < ^k cos ipk<- cos (pk + \Jl-S.l, 

in which the upper bound is negative provided that — cos (pk + ~ ^k < 0; which means 
^fc > smcpk- So cos ipk and cos (pk must have opposite signs if ^k > sin (pk- Hence, it follows 
from dMI that ^ holds if ^k > sm(pk. Combining (HH) with ([IT]) and ^ gives ([26]). □ 

Since Uk is assumed to be a reasonably good approximation to x, siiKpk is small. As a 
result, the condition ^k > sin (pk is easily satisfied unless the linear system is solved with 
very high accuracy. 

Clearly, how general this theorem depends on how general assumption ()2ip is. Next 
we give a qualitative analysis to show that this assumption is very reasonable and holds 
generally, indicating that the theorem is of rationale and generality. 

Note that by definition we have 

Cfc and fk £ span{x2, X3, . . . , x^}, 

so does 

{A - 9kl)ek G span{x2, X3, . . . , x„}. 

We now justify the generality of and fk- In the proof of cubic convergence of RQI, which 
is the inexact RQI with ^k = 0, Parlett [151 P- 78-79] proves that will start to converge 
to X2 only after Uk has converged to x and — )• X2 holds for large enough k. In other 
words, Ck is a general combination of X2, X3, • • • , and does not start to converge before Uk 
has converged. Following his proof path, we have only two possibilities on in the inexact 
RQI with MINRES: One is that e^, at best, can possibly start to approach X2 only if Uk has 
converged to x; the other is that is nothing but just still a general linear combination of 
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X2,X3, ■ ■ ■ ,Xn and does not converge to any specific vector for any ^fc- In either case, is 
indeed a general linear combination of X2, ^3, . . . , x„ before has converged. 
Expand the unit length as 

n 

with X^"=2 '^j ~ I- Then, based on the above arguments, no aj is small before Uk has 
converged. Note that 

n 

{A - 0kl)ek = '^aj{Xj - 9k)xj. 

Since 9^ supposed to be a reasonably good approximation to A, for j = 2, 3, . . . , n, — 9k 
are not small, so that (A — 9kl)ek is a general linear combination of 2:2, 2:3, . . . , Xn- 

Let Pm{z) be the residual polynomial of MINRES applied to Then it is known [13] 
that Prn(O) = 1, its m roots are the harmonic values of A — 6^1 with respect to ICm{A, u^). 
From pop and (|11|) we can write the residual ^kdk as 

ikdk = Pm{A- 9kl)uk = Pm{A- 9kl){x cos (l)k + Cksincpk) 

n 

= COS (pkPm{\ - 9k)x + sin (jjk'^ a jPm{\j - 9k)xj 

i=2 

= ^fc(xcosV'fc + /fcsin■i/'fc)• 
Noting that WfkW = 1, we get 

^ p„,{A-ekI)ek ^ E"=2 ajPmjXj - 9k)xj 
WpUA - 9kl)ek\\ iE]=2 c^jpli^j - • 

Since Uk is rich in x and has small components in 2:2, X3, . . . ,Xn, ICmiA,Uk) contains not 
much information on 2:2, X3, . . . , x„ unless m is large enough. Therefore, as approximations 
to the eigenvalues A2— ^fci -^s^^fcj • • • > -^n— ^fc of the matrix A—9kI, the harmonic Ritz values 
are generally of poor quality unless m is large enough. By continuity, Pmi^j — 9k), j = 
2,3, ... ,n are generally not near zero. This means that usually fk is a general linear 
combination of X2,X3, . . . ,Xn, which is the case for ^k not very small. We point out that 
Pmi^j — 9k), j = 2, 3, n can be possibly near zero only if is small enough, but in this 
case the cubic convergence of the outer iteration can be established trivially, according to 
Theorem [21 For other discussions on S^k and harmonic Ritz values, we refer to [22]. 

In view of the above, it is very unlikely for fk and {A — 9kl)ek to be nearly orthogonal, 
that is, ifk is rarely near ^. So, | cos^pkl should be uniformly away from zero in general, 
and assumption (j2ip is very general and reasonable. 

Obviously, | costpkl is a-priori and cannot be computed in practice. For test purposes, 
for each matrix in Section [5] and some others, supposing the x's are known, we have com- 
puted |cosv9;j| for = 0(||rfc||), Cfc < ^ < 1 with ^ a fixed constant, Cfc = 1 ~ (Ikfcll) 
and Cfc = 1 — 0(||rfc|p), respectively. As will be seen, the latter three requirements on ^k 
are our new cubic, quadratic and linear convergence conditions for the inexact RQI with 
MINRES that will be derived by combining this theorem with Theorems [2H3] and are crit- 
ically need the assumption that |cos99fc| is uniformly away from zero, independent of k. 
Among thousands |cos(^fc|'s, we have found that most of the |cos(^fc|'s are considerably 
away from zero and very few smallest ones are around 10"''. Furthermore, we have found 
that their arithmetic mean is basically 0.016 ~ 0.020 for each matrix and a given choice 
of (,k above. To highlight these results and to be more illustrative, we have extensively 
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computed | cosZ(w,v)\ = with many vector pairs {w,v) of nmnerous dimensions no 

less than 1000 that were generated randomly in a normal distribution. We have observed 
that the values of our |cos(/3fc|'s and their arithmetic mean for each matrix and a given 
choice of above have very similar behavior to those of thousands | cos /(tD, t>)|'s and 
the arithmetic mean for a given dimension. As a consequence, this demonstrates that as- 
sumption (I2ip are very like requiring the uniform non-orthogonality of two normal random 
vectors and thus should hold generally. In other words, fk,ek and {A — 9kl)e-k are usually 
indeed general linear combinations of X2 , • • • , x„ and have the nature as vectors generated 
in a normal distribution, so assumption (12 ip should hold generally. On the other hand, our 
later numerical experiments also confirm the cubic, quadratic and linear convergence under 
our corresponding new conditions This means that the numerical experiments also justify 
the rationale and generality of the assumption a-posteriori. 

Based on Theorem [4] and Theorems [SHSJ we now present new convergence results on the 
inexact RQI with MINRES. 

Theorem 5. With cosy? defined as in Theorem^ assuming that \ cosy?! ^■^ uniformly away 
from zero, the inexact RQI with MINRES asymptotically converges cuhically: 

2/3(1 cos (^|+2gfc/3) . 3 
tan (/>fc+i < — ^ sm^ (jy^ (29) 

if ^ i with a fixed ^ not near one; it converges quadratically: 

tan</)fc+i < rjsiv? (pk (30) 

if ik ^-5 near one and hounded by 

4<1-^^^ (31) 
rji cos ip\ 

with 7] a moderate constant; it converges linearly at least: 

tan(/)fc+i < Csin</>fc (32) 
with a constant C < 1 independent of k if is bounded by 

6/32 sin 0fc 6/32 sin2 

1 , r <l -r. p- (33) 

r/|cos(/9| CI cosy?! 

Proof. Based on Theorem U we have 

cos (j)k+^k cos il^k = 1 - ^sin2</'fc ±Cfc(l - ©(sin^^fc)) 
= l±Ck + 0{sm^cPk) 

= l±^k>l-^k (34) 

by dropping the higher order term ©(sin^ (pk). Therefore, the uniform positiveness condition 
holds provided that 4fc < ^ with a fixed ^ not near one. Combining (fTCj) with ([25]) and ([M])j 
we get 

l + CkT^ 

1 I ^ r\ n ' \ COS (iP . q , 

tan0fc+i<2/3 — sm (pk, 

1 - ?fc 

which is just (j29p and shows the cubic convergence of the inexact RQI with MINRES if 
^fc < C with a fixed ^ not near one. 
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Next we prove the quadratic convergence result. Since ^a: < 1 and (3 > 1, it follows from 
23|) and ([M|) that asymptotically 

sin 0fc + sin jjk ^ (1 + Wl cos sin 0^ 
I cos (t)k + Cfc cos i>k I cos + ik cos V'fc 

6/3^ sin (j)k 



< 



< 



(cos + ik COS V'fc) I COS 99| 

6/3^ sin (j)k 



(1 -Cfc)|cos(/?|' 

So from (jlSp the inexact RQI with MINRES converges quadratically and ()30p holds if 

6/3^ sin (^fc 



(1 - cosv?! 



< 



for a moderate constant r/ independent of A;. Solving this inequality for ^k gives ([31 
Finally, we prove the linear convergence result. Analogously, we have 

sin + ^fc sin -i/^fc . 6/3^ sin^ 0^. 

^P'\ 1 ? rrsmcpfc < 



COS^fe +CfcCOsV'fc| (1 -^fc)|cOS99|' 

So it follows from (jl5p that the inexact RQI with MINRES converges linearly at least and 
(j32]l holds when 

6/3^sin^0fc ^^^^ 

(1 - COS(/7| 

with a constant C independent of k. Solving the above inequality for £^k gives ([33]) . □ 

Theorem [5] presents the convergence results in terms of the a priori uncomputable sin (pk- 
We next derive their counterparts in terms of the a posteriori computable \\rk\\, so that 
they are of practical value as much as possible and can be used to best control inner-outer 
accuracy to achieve a desired convergence rate. 

Theorem 6. With cos ip defined as in Theorem^ assuming that \ cos(p\ is uniformly away 
from zero, the inexact RQI with MINRES asymptotically converges cuhically: 

16/3^(1 cos c^|+2e./3) 3 .35^ 
- (1 -C.)(A2- API cose,! (''^ 

if ^ C with a fixed ^ not near one; it converges quadratically: 

II II ^ II l|2 (oa\ 

lkfc+i||<-- -jllrfcll (36) 

|A2 — A| 

if is near one and hounded by 



ik<i- |, I (37) 

r]\X2 - \\ \ cosLp\ 

with rj a moderate constant; it converges at linear factor ( at least: 

||rfc+i|| < Clkfcll (38) 

if S,k is bounded by 



^ _ -^-lkfcll < < 1 - ^^/^^ll^fcll' (39) 

?7|A2 - A|| COS(^| ~ C(A2 - A)2| cos 



with a constant C < 1 independent of k. 
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Proof. From psp we have 



kfc+ill 2||rfc| 
< sin^fc+i < tan(j)k+i, sin^fc < 



^max ~ ^min 1^2 ~ ^| 

So (|35|) is direct from (|29|) by a simple manipulation. Next we use (|18|) to denote sin ( 
with ^^^2 < C* < Amax — -^min- Note that 

sin + gfc sin ipk ^ 6(3'^sm(l)k QP'^\\rk\\ 



>k 



cos (f)k+^k cos ll^kl (1 - Cfc)| COS((i?| C(l - ^fc)| cosv^l 

Therefore, similarly to the proof of Theorem [5l if 

C{1 - Ck)\ coscpl 

the inexact RQI with MINRES converges quadratically. Solving this inequality for gives 

6/32||rfc|| , , 

a < 1 - " "i - 40 

C?7| cos 



Note that 



6/32||rfc|| ^ ^ 6/32||rfc| 



C??! cos (p\ ??(Amax - Amin) | COS (p\ 

^ m\rk\\ 

rj\X2 - A|| cos (p\ 

So, if ^fc satisfies (j37p . then it satisfies (j40p too. Therefore, the inexact RQI with MIN- 
RES converges quadratically if ()37p holds. Furthermore, from (jlSp we have ^j2-x ■ ^ 
sin</>fc_|_i < tan0fc_|_i. As a result, from (j30p we obtain 

ll'^fe+lll ^ (Amax ~ Aniin)^- 



C2 



proving (|36|) . 

In order to make ||rfc|| monotonically converge to zero linearly, by ([35]) we simply set 



16/3^(|cosv?|+2efc/3)||rfcf ^ A8f3^rk\\^ ^ 

(l-6)(A2-A)2|cos(/?| - (1-6)(A2- A)2|cos(p| 

with independent of k. Solving it for gives 

48/33||rfc||2 



C(A2 — A)2| cos V9| 

Combining it with §7^ proves dMD and ([39]). □ 



First of all, we make a comment on Theorems [SHSl As justified previously, assump- 
tion (j2ip is of wide generality. Note that Theorems [2H3] hold as we always have cos 4>k + 
COS V'fc > 1 — Cfc asymptotically, independent of fk- Therefore, in case | cos if\ is occasion- 
ally near and even zero, the inexact RQI with MINRES converges quadratically at least 
provided that ^^fc < ^ with a fixed not near one. 
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In order to judge cubic convergence quantitatively, we should rely on Theorem [2] and 
Theorem [5] (equivalently, Theorem [3] and Theorem [6]) , in which cubic convergence precisely 
means that the asymptotic convergence factor 



sin (pk+i 
sin^ (pk 



< 2/3 



(41) 



for RQI and the asymptotic convergence factor 



sin(/)fc+i ^ 2/3(1 cos 01 +2efc/3) 
sin^^fc ~ (1 - Cfc)l cosv9| 



2/3 , 4^^/32 



(42) 



(1 - COS(y9| 



for the inexact RQI with MINRES. The asymptotic factors in (j4ip - (j42p do not affect the 
cubic convergence rate itself but a bigger asymptotic factor affects reduction amount at 
each outer iteration and more outer iterations may be needed. 

Some other comments are in order. First, the bigger /3 is, the bigger the asymptotic 
convergence factors are and meanwhile the considerably bigger the factor in (|42|) is than 
that in (j4ip if is not near zero. In this case, the cubic convergence of both RQI and the 
inexact RQI may not be very visible; furthermore, RQI may need more outer iterations, 
and the inexact RQI may need more outer iterations than RQI. Second, if the factor in ()4ip 
is not big, the factor in (j42p differs not much with it provided ^ ^ < ^ with a fixed ^ not 
near one, so that the inexact RQI and RQI use (almost) the same outer iterations. Third, 
it is worth reminding that the factor in (j42l) is an estimate in the worst case, so it may 
often be conservative. Fourth, as commented previously, the inexact RQI with MINRES 
may behave more like quadratic in case | cos(/9| is occasionally very small or even zero. 

Theorem [6] shows that ||rfc|| can be used to control in order to achieve a desired 
convergence rate. For cubic convergence, at outer iteration k we only need to solve the 
linear system ([1]) by MINRES with low accuracy < It is safe to do so with ^ = 0.1, 0.5 
and even with ^ = 0.8. A smaller ^ is not necessary and may cause much waste at each outer 
iteration. Thus, we may save much computational cost, compared with the inexact RQI with 
MINRES with decreasing tolerance = 0(sin(/)fc) = 0(||rfc||). Compared with Theorem [H 
another fundamental distinction is that the new quadratic convergence results only require 
to solve the linear system with very little accuracy Cfc = 1 ~ 0(sin(/)fc) = 1 — 0(||rfc||) 1 
rather than with ^fc < ^ not near one. They indicate that the inexact RQI with MINRES 
converges quadratically provided that coscpk + ^k^^osipk w 1 — ^fc = 0(sin(/)fc) = 0(||rfc||). 
The results also illustrate that the method converges linearly provided that Cfc = 1 ~ 
0(sin^ (pk) = 1 — 0(||rfc|p). In this case, we have cos(pk + Ca: costpk ~ 1 — Cfc = 0(sin^ (pk) = 
0(||rfc|p). So can be increasingly closer to one as the method converges when quadratic 
and linear convergence is required, and can be closer to one for linear convergence than 
for quadratic convergence. These results make it possible to design effective criteria on 
how to best control inner tolerance in terms of the outer iteration accuracy ||rfe|| to 
achieve a desired convergence rate. In addition, interestingly, we comment that, instead 
of quadratic convergence as the authors of [T] claimed. Table 1 in [1] actually showed the 
same asymptotic cubic convergence of the inexact RQI with MINRES for a fixed = 0.1 
as that for decreasing tolerance = 0(||rfc||) in the sense of (l4T]l and (l42]l . 

Below we estimate in ([7|) obtained by MINRES and establish a lower bound for 

it. As a byproduct, we also present a simpler but weaker quadratic convergence result. Note 
that the exact solution of {A — 6kl)w = is w^+i = {A — 6kI)~^Uk, which corresponds to 
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^fc = in (fT3l) . Therefore, with a reasonably good Ufc, from (fT3l) . (fT7|l and (fTSl) we have 

II II COS(j)k , ^/ . , s 



\0k - A| 



Theorem 7. Asymptotically we have 



Thus, the inexact RQI with MINRES converges quadratically at least once is not near 
one. 



Proof. By using ((TSD, ([T8]) and ([Ml in turn, we obtain 

I cos <l)k +ik COS V^fcl 

\ek - A| 

I COS (f)k+^k COS 1pk\ 
(Amax — Amin) sin^ 
I COS(j)k +Ck COsV>A:||A2 - A| 

4/3||rfc||2 
(l-a)|A2-A| + 0(sin2(/)fc) 

4/?||rfcP 
(l-efc)|A2-A| 
4/3||rfcP 

with the last equality holding asymptotically by ignoring 0(1). This proves ([13 
It follows from (j28|) and Ufc+i = ^^'fc+i/ll'U^fc+i || that 



> 

> 
> 



\\{A-eki)uk+i\\ " " 

\\Wk+l\ 

So from the optimality of Rayleigh quotient we obtain 



\rk+i\\ = ||(^ - 9k+il)uk+i\\ < ||(^ - ekl)uk+i\\ = J. (45) 

ll^fc+ill 

Substituting (03]) into it establishes (jS]). □ 
Simoncini and Elden [T7] present an important estimate on 

|1 — Eml COS^ <^fc 1 

ll'»^fc+i|| > ^— 71 — H"' (46) 

sm0fc ||rfc|| 

where £m = Pm(A — 9k) with pm the residual polynomial of MINRES satisfying Pm{0) = 1; 
see Proposition 5.3 there. This relation involves £m and is less easily interpreted than 
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(1431) . When Em is not near one, is bounded by Oi-rr-Kyi) from below. Based on this 

\rk II 

estimate, Simoncini and Elden have designed a stopping criterion for inner iterations. 
From (|43p and Theorems [SHHl it is instructive to observe the remarkable facts: Hwfc+iH 

increases as rapidly as Oi ,, ^u-A and Oi-rr^), respectively, if the inexact RQI with MINRES 

iKfcll ii^fcii 

converges cubically and quadratically; but it is 0(1) if the method converges linearly. As 
(P3|) is sharp, the size of Hwfc+iH can reveal both cubic and quadratic convergence. We can 
control to make the method converge cubically or quadratically, as done in |17j . 

However, it is unlikely to do so for linear convergence as the method may converge linearly 
or disconverges when ||i(7fc_|_i|| remains 0(1). 

Another main result of Simoncini and Elden |17j is Proposition 5.3 there: 



sm(/)fc V" 

rfc+i < T7rT\ \Wk\V (47) 

COS'^(/)i, 1 - Em, 



Note sin (f)k = 0(||rfc||). The above result means quadratic convergence if ] is moderate, 
which is the case if and Em are not near one. We refer to [131 [22] for discussions on Em- 
Since how and affect each other is complicated, ()44p is simpler and more easily 
understandable than (j47p . However, both (j44p and (j47p are weaker than Theorems [SHS] 
since quadratic convergence requires .^^ < 1 not near one and the cubic convergence of RQI 
and of the inexact RQI cannot be recovered when = and = 0(||rfc||), respectively. 



4 The inexact RQI with a tuned preconditioned MINRES 

We have found that even for near one we may still need quite many inner iteration 
steps at each outer iteration. This is especially the case for difficult problems, i.e., big /3's, 
or for computing an interior eigenvalue A since it leads to a highly Hermitian indefinite 
matrix [A — Okl) at each outer iteration. So, in order to improve the overall performance, 
preconditioning may be necessary to speed up MINRES. Some preconditioning techniques 
have been proposed in, e.g., [HITT]. In the unpreconditioned case, the right-hand side Uk of 
dH) is rich in the direction of the desired x. MINRES can benefit much from this property 
when solving the linear system. Actually, if the right-hand side is an eigenvector of the 
coefficient matrix, Krylov subspace type methods will find the exact solution in one step. 
However, a usual preconditioner loses this important property, so that inner iteration steps 
may not be reduced [H HI E] . A preconditioner with tuning can recover this property and 
meanwhile attempts to improve the conditioning of the preconditioned linear system, so 
that considerable improvement over a usual preconditioner is possible [H [U [22]. In this 
section we show how to extend our theory to the inexact RQI with a tuned preconditioned 
MINRES. 

Let Q = LL* be a Cholesky factorization of some Hermitian positive definite matrix 
which is an approximation to A — Otl in some sense [U [SI [22]. A tuned preconditioner 
Q = CC* can be constructed by adding a rank-1 or rank-2 modification to Q, so that 

Quk = Auk] (48) 

see m [5l [22] for details. Using the tuned preconditioner Q, the shifted inner linear system 
dl]) is equivalently transformed to the preconditioned one 

Bw = C~^{A - 9kI)C-*w = C-^Uk (49) 

with the original w = C'~*w. Once MINRES is used to solve it, we are led to the inexact RQI 
with a tuned preconditioned MINRES. A power of the tuned preconditioner Q is that the 
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right-hand side C~^Uk is rich in the eigenvector of B associated with its smahest eigenvalue 
and has the same quahty as as an approximation to the eigenvector x of A, while for 
the usual preconditioner Q the right-hand side L~^Uk does not possess this property. 

Take the zero vector as an initial guess to the solution of (09]) and let Wk+i be the 
approximate solution obtained by the m-step MINRES applied to it. Then we have 

£-\A - ekI)C-*Wk+i = C-^Uk + ikdk, (50) 

where Wk+i G ICm{B, C'^Uk), Ckdk with \\dk\\ = 1 is the residual and dk is the residual 
direction vector. Trivially, for any m, we have ^k ^ Keep in mind that Wk+i = 

C~*Wk+i- We then get 

{A-9kI)wk+i = Uk + ik^dk = Uk + ik\\^dk\\ . . (51) 

ll^«fc|| 

So ^k and dk in (171) are and 777x17, respectively, and our general Theorems [2H3] 

ll^"fell 

apply and is not repeated. In practice, we require that the tuned preconditioned MINRES 
solves the inner linear system with < l/||£(ifc|| such that < 1. 

How to extend Theorem |3]to the preconditioned case is nontrivial and needs some work. 
Let (/ij, yj), i = 1, 2, . . . , n be the eigenpairs of B with 

I /^l |<| At2 |< • • • <| /^n I • 

Define % = i^~^Uk/\\C-~'^Uk\\- Similar to (fTOl) and ([TT]) . let 

ilk = yicos(f)k + eksmc()k, ik ±yi, \\ek\\ = I, (52) 
4 = yicosV^fc -h /fcsinVifc, A -L yi, ll/fcll = 1 (53) 

be the orthogonal direct sum decompositions of Uk and dk- Then it is known [5] that 

|/ii| = 0{sm(pk), (54) 
sin (pk < ci sin (pk (55) 

with ci a constant. 

Based on the proof line of Theorem |4] and combining ()55p , the following results can be 
proved directly. 

Lemma 1. For the tuned preconditioned MINRES, let the unit length vectors Ck and fk be 
as in (j52p and (153p . define the angle (pk = Z{fk, Bik) and assume that 

I cos (^fc| > I cos > (56) 

holds uniformly for an angle (p away from ^ independent of k. Then we have 

sin-iAfc < 7^^"^ sin(/>fc, (57) 

cosV^fc = ±(l-0(sin2(/,fc)), (58) 
dk = ±yi + 0{sm(l)k)- (59) 

We can make a qualitative analysis, similar to that done for Theorem [4] in the unpre- 
conditioned case, and justify the assumption on ipk. With this theorem, we now estimate 
sin ijjk and cos V'fc • 
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Theorem 8. For the tuned preconditioned MINRES, under the assumptions of Theorem\^ 
it holds that 

sintpk = O(sin0fc), (60) 
cosV'fc = ±{l - 0{W (t>k)) (61) 
4 = =±x + 0(sin(/>fc). (62) 

Proof. By definition, we have 

C-\A-9kI)C-*yi = iiiyi, 

from which it follows that 

{A - ekI)Uk = m 777%— FT 

yiW 

with Uk = C~*yi/\\C^*yi\\. Therefore, by standard perturbation theory and (f54|l . we get 

\ \ r II 

sinZ(nfc,x) = 0{^ii ) = 0(|/Ui|) = 0(sin(/>fc). 

11-^ yiii 



Since 
we get 



^(-"fc, Uk) < ^{uk,x) + Z{uk,x), 
sin Z{uk,Uk) = 0{sm<j)k), 



I.e., 



Uk = Uk + O {sin 4>k)- 
As a result, from Auk = Quk and Uk = *yi/||'^ *yi|| we have 

Cyi = \\C^*yi\\/:C*Uk = \\C'*yi\\Quk 
= ||£"*yi||Q('Ufc + 0(sin(/)fc)) 
= ||£"*yi||^nA,. + 0(sin(^fc) 
= \\^~*yi\\{OkUk+rk) +0{sin(l}k) 
= ek\\C~*yi\\uk + \\C~*yi\\rk + 0{sin<j)k), 

from which we obtain 

sinZ{Cyi,Uk) = 0{\\rk\\) + O(sin0fc) = 0{sin(pk)- 

Therefore, we have 

sin Z(£yi, x) < sinZ{Cyi,Uk) + sin Z{uk,x) = 0(sin(/)fc), (63) 

that is, the (unnormalized) Cyi has the same quality as Uk as an approximation to x. We 
have the orthogonal direct sum decomposition 

Cyi = \\Cyi\\ (xcosZ(£yi,x) + gk sin Z{Cyi, x)) (64) 

with \\gk\\ = 1 and gk -L x. Also, make the orthogonal direct orthogonal sum decomposition 

Cfk = \\Cfk\\ ( X cos Z{Cfk, x) + hk sin Z{Cfk,x)) (65) 
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with \\hk\\ = 1 and hk -i- x. Making use of (f57|l and (i58]l . we get from (|53]l that 



Hence we have 



\Ckdk\\ = \\Cyi\\ cos Vfc + II^C/fcll sin tpk + 2Re{Cyi, Cfk) sinipk cos ipk 
= \\jCyif + 0{sm^(t>k) + 0{sm^k) 
= + 0(sin(/)fc). 

\\Cyi\\ = \\Cdk\\+0{sm^k)- (66) 



Now, from (153p . we obtain 



^'^^ = 771^7 = II A (£yicosV^fc + £/fcsinV'fc)- 
||£4|| \\J~-dk\\ 

Substituting and into the above relation yields 

dk = —-^{{\\Cyi\\cosZ{Cyi,x)cosi)k + \\Cfk\\cosZ{£fk,x)sm'il^k)x 
W^dkW 

+{\\Cyi\\gk sinZ(£yi,x) cos ipk + \\Cfk\\hk sin Z{Cfk,x) sinipk))- 

This is the orthogonal direct sum decomposition of dk- Making use of sinZ(>Cyi,x) = 
0(sin(/)fc), sinV'fc = O {sin cpk) and (f66|) and comparing with dk = xcostpk + sinV'fc, we get 

sinV'fc = Oisincpk)- 

Thus, it holds that cos V'fc = i(l — 0(sin^ (pk)) and dk = ±2; + 0{sin(j)k)- D 

Compared with Theorem [H this theorem indicates that the directions of residuals ob- 
tained by the unpreconditioned MINRES and the tuned preconditioned MINRES have the 
same properties. With the theorem, we can write sin ipk < C2sin0fc with C2 a constant. 
Then, based on Theorems [2H3l it is direct to extend Theorems [5H6] to the inexact RQI with 
the tuned preconditioned MINRES, respectively. We have made preliminary experiments 
and confirmed the theory. Since our main concerns in this paper are the convergence theory 
and a pursue of effective tuned preconditioners are beyond the scope of the current paper, 
we will only report numerical experiments on the inexact RQI with the unpreconditioned 
MINRES in the next section. 



5 Numerical experiments 

Throughout the paper, we perform numerical experiments on an Intel (R) Core (TM)2 
Quad CPU Q9400 2.66GHz with main memory 2 GB using Matlab 7.8.0 with the machine 
precision emach = 2.22 x 10~^^ under the Microsoft Windows XP operating system. 

We report numerical experiments on four symmetric (Hermitian) matrices: BCSPWR08 
of order 1624, CAN1054 of order 1054, DWT2680 of order 3025 and LSHP3466 of order 
3466 [3J. Note that the bigger /3 = '^"j^'^"^™'" is, the worse conditioned x is. For a bigger 
f3, Theorem [2] and Theorems [5H6] show that RQI and the inexact RQI with MINRES may 
converge more slowly and use more outer iterations though they can still converge cubically. 
If an interior eigenpair is required, the shifted linear systems can be highly indefinite (i.e., 
many positive and negative eigenvalues) and may be hard to solve. As a reference, we use 
the Matlab function eig.m to compute /3. To better illustrate the theory, we compute both 
exterior and interior eigenpairs. We compute the smallest eigenpair of BCSPWR08, the 
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tenth smallest eigenpair of CAN1054, the largest eigenpair of DWT2680 and the twentieth 
smallest eigenpair of LSHP3466, respectively. 

Remembering that the (asymptotic) cubic convergence of the inexact RQI for = 
0(||rfc||) is independent of iterative solvers, in the experiments we take 

Ck = min{0.1, Jf^}. (67) 
ll^lli 

We first test the inexact RQI with MINRES for ^fc ^ C < 1 with a few constants 
^ not near one and illustrate its cubic convergence. We construct the same initial uq 
for each matrix that is x plus a reasonably small perturbation generated randomly in a 
uniform distribution, such that |A — ^ol < ^-5^- The algorithm stops whenever ||r/;|| = 
||(yl — 0jt/)ufc|| < ||yl||itoZ, and we take tol = IQ-^^ unless stated otherwise. In experiments, 
we use the Matlab function minres.m to solve the inner linear systems. Tables [iHll list 
the computed results, where iters denotes the number of total inner iteration steps and 
^^gj,(fc-i) jg ^]^g number of inner iteration steps when computing {9k,Uk), the "-" indicates 
that MINRES stagnates and stops at the iteA''~^^-th step, and res^''~^^ is the actual relative 
residual norm of the inner linear system when computing {6k, Uk)- Clearly, iters is a 
reasonable measure of the overall efficiency of the inexact RQI with MINRES. We comment 
that in minres.m the output iter^^~^^ = m — 1, where m is the steps of the Lanczos process. 
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^ Pill J 

tol = 10-1° 


1 (1); 2 (3); 3 (9); 4 (13); 5 (15); 6 (16) 
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Table 1: BCSPWR08, (3 = 40.19, sin</.o = 0.1134, ci = C2 = 1000. k (iter^^-i)) denotes 
the number of inner iteration steps used by MINRES when computing {Ok,Uk)- 

Before explaining our experiments, we should remind that in finite precision arithmetic 
||rfc||/||^||i cannot decrease further whenever it reaches a moderate multiple of emach = 
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Table 2: CAN1054, /? = 88.28, sm(/>o = 0.1137, ci = C2 = 1000. /c (iter^'^-i)) denotes the 
number of inner iteration steps used by MINRES when computing {6k,Uk)- 



2.2 X 10- . Therefore, assuming that the algorithm stops at outer iteration k, if sin(/)fc_i 
or ||rfc_i|| is at the level of 10-^ or 10-^, then the algorithm may not continue converging 
cubically or quadratically at the final outer iteration k. 

To judge cubic convergence, we again stress that we should rely on (j4ip and ()42p for 
RQI and the inexact RQI with MINRES, respectively. We observe from the tables that the 
inexact RQI with MINRES for 6 ^ C < 1 with ^ fixed not near one converges cubically and 
behaves like RQI and the inexact RQI with MINRES with decreasing tolerance 6 = (lkfc||)j 
it uses (almost) the same outer iterations as the latter two do. The results clearly indicate 
that cubic convergence is generally insensitive to ^ provided ^ is not near one. Furthermore, 
we see that the algorithm with a fixed ^ not near one is much more efficient than the 
algorithm with = 0(||rfc||) and is generally about one and a half to twice as fast as the 
latter. Since {A — 9kl)w = Uk becomes increasingly ill conditioned as k increases, we need 
more inner iteration steps to solve the inner linear system with the same accuracy 6 though 
the right-hand side Uk is richer in the direction of x as /c increases. For = 0{\\rk\\), inner 
iteration steps needed can be much more than those for a fixed not near one at each outer 
iteration as k increases. We refer to fl] for a descriptive analysis. 

For the above numerical tests, we pay special attention to the ill conditioned DWT2680 
and LSHP3466. Intuitively, RQI and the inexact RQI with MINRES seems to exhibit 
quadratic convergence. However, it indeed converges cubically in the sense of (j4ip and 
(|42p . With ^ = 0.5, sin(/)fc and ||rfc|| decrease more slowly than those obtained with ^ = 0.1 
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Table 3: DWT2680, tol = 10"^^ (3 = 2295.6, sin(/.o = 0.1133, ci = 10000, C2 = 1000. 
k {iter^^~^^) denotes the number of inner iteration steps used by MINRES when computing 
{Ok,Uk)- 



and the exact RQI as well as the inexact RQI with decreasing tolerance, and the algorithm 
uses one more outer iteration. This is because the convergence factors in both (j4ip and 
(|42p are big and the factor with ^ = 0.5 is considerably bigger than those with the others. 
However, the method with ^ = 0.5 uses comparable iters. 

Our experiments show that the inexact RQI with MINRES is not sensitive to < 1 not 
near one. So it is advantageous to implement the inexact RQI with MINRES with a fixed 
not near one so as to achieve the cubic convergence. We can benefit much from such a 
new implementation and use possibly much fewer iters, compared with the method with 
^k = 0{\\r,\\). 

Next we confirm Theorems [5H6] and verify quadratic convergence and linear convergence 
when conditions (j37|) and (f39|) are satisfied, respectively. Note that /3 and |cos(/?| in the 
upper bounds for are uncomputable a priori during the process. However, by their forms 
we can take 

6 = 1- (68) 
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Table 4: LSHP3466, tol = 10-l^ p = 2613.1, sinc/^o = 0.1011, ci = C2 = 1000. k (iter^^-^)) 
denotes the number of inner iteration steps used by MINRES when computing {9k,Uk)- 



and 



\A\ 



6 = 1-(S^) (69) 



for reasonable ci and C2 , respectively, and use them to test if the inexact RQI with MINRES 
converges quadratically and linearly. It is seen from ([37]) and ([39|) that we should take ci 
and C2 bigger than one as /3 > 1, | cos(p\ < 1 and C < 1- The bigger /3 is, the bigger ci 
and C2 should be. Note that defined so may be negative in the very beginning of outer 
iterations if uq is not good enough. In our implementations, we take 

a = max{0.95, 1 - (70) 



efc = max{0.95,l-(^|^)'} (71) 



and 

= max-rn.95. 1 - I - 

. I^ll 

with 100 < ci,C2 < 10000 for quadratic and linear convergence, respectively. As remarked 
previously, the inexact RQI with MINRES for ^ = 0.8 generally converges cubically though 
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it may reduce \\rk\\ and sini;^^ not as much as that for ^ smaller at each outer iteration. We 
take it as a reference for cubic convergence. We implement the method using ()68p and (j69p . 
respectively, after very few outer iterations as long as the algorithm starts converging. They 
must approach one as outer iterations proceed. Again, we test the above four matrices. In 
the experiments, we have taken several ci,C2's ranging from 100 to 10000. The bigger ci 
and C2 are, the safer are bounds (I70p and (I7ip for quadratic and linear convergence, and 
the faster the algorithm converges. We report the numerical results for ci = C2 = 1000 in 
Tables [IHH except ci = 10000 for DWT2680. Figure [1] draws the convergence curves of the 
inexact RQI with MINRES for the four matrices for the fixed ^fc = 0.8 and ci = C2 = 1000 
except ci = 10000 for DWT2680. 





outer iterations 



outer iterations 



Figure 1: Quadratic and linear convergence of the inexact RQI with MINRES for BC- 
SPWR08, CAN1054, DWT2680 and LSHP3466 in order, in which the sohd line denotes 
the convergence curve of = ^ = 0.8, the dotted dash line the quadratic convergence curve 
and the dashed line the linear convergence curve. 

Figure [1] clearly exhibits the typical behavior of quadratic and linear convergence of 
the inexact RQI with MINRES. Precise data details can be found in Tables [THll As outer 
iterations proceed, is increasingly closer to one but the algorithm steadily converges 
quadratically and linearly; see the tables for quadratic convergence. The tables and figure 
indicate that our conditions (I70p and (I7ip indicate the inexact RQI works very well for 
chosen ci and C2. For quadratic convergence, becomes increasingly closer to one, but on 
the one hand iteA^~^'> still increases as outer iterations proceed and on the other hand it is 
considerably smaller than that with a fixed ^. In contrast, for linear convergence, iter^^~^^ 
varies not much with increasing k except for the first two outer iterations, where iter^^^^^ 
is no more than five. 

For other ci and C2, we have made experiments in the same way. We have observed 
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similar phenomena for quadratic convergence and found that the algorithm is not sensitive 
to ci in general, but this is not the case for C2- For different C2, the method still converges 
linearly but the number of outer iterations may vary quite a lot. This should be expected 
as C2 critically affects the linear convergence factor C, that uniquely determines convergence 
speed, while ci does not affect quadratic convergence rate and only changes the factor rj 
in the quadratic convergence bounds (f30]) and ([361) . Also, we should be careful when using 
(j7ip in finite precision arithmetic. If 



is at the level of e or smaller for some then ()7ip gives = 1 in finite precision arithmetic. 
The inexact RQI with MINRES will break down and cannot continue the {k + l)-th outer 
iteration. A adaptive strategy is to fix to be a constant smaller than one once ||rfc|| is so 
small that = 1 in finite precision arithmetic. We found that ^fc = 1 — 10~® is a reasonable 
choice. We have tested this strategy for the four matrices and found that it works well. 

6 Concluding remarks 

We have considered the convergence of the inexact RQI without and with MINRES in 
detail and have established a number of results on cubic, quadratic and linear convergence. 
These results clearly show how inner tolerance affects the convergence of outer iterations 
and provide practical criteria on how to best control inner tolerance to achieve a desired 
convergence rate. It is the first time to appear surprisingly that the inexact RQI with 
MINRES generally converges cubically for ^fc < ^ < 1 with ^ a constant not near one and 
quadratically for increasingly near one, respectively. They are fundamentally different 
from the existing results and have a strong impact on effectively implementing the algorithm 
so as to reduce the total computational cost very considerably. 

Using the same analysis approach in this paper, we have considered the convergence of 
the inexact RQI with the unpreconditioned and preconditioned Lanczos methods for solving 
inner linear systems [9], where quadratic and linear convergence remarkably allows ^^a; ^ 1 
considerably, that is, approximate solutions of the inner linear systems have no accuracy 
at all in the sense of solving linear systems. By comparisons, we find that the inexact RQI 
with MINRES is preferable in robustness and efficiency. 

Although we have restricted to the Hermitian case, the analysis approach could be 
used to study the convergence on the inexact RQI with Arnoldi and GMRES for the non- 
Hermitian eigenvalue problem. 

We have only considered the standard Hermitian eigenvalue problem Ax = Xx in this 
paper. For the Hermitian definite generalized eigenvalue problem Ax = XMx with A 
Hermitian and M Hermitian positive definite, if the M-inner product, the M-norm and the 
M~^-norm, the angle induced from the M-inner product are properly placed in positions of 
the usual Euclidean inner product, the Euclidean norm and the usual angle, then based on 
the underlying M-orthogonality of eigenvectors of the matrix pair (A, M), we should be able 
to extend our theory developed in the paper to the inexact RQI with the unpreconditioned 
and tuned preconditioned MINRES for the generalized eigenproblem. This work is in 
progress. 
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